The present invention relates to the art of diagnostic imaging, and, in particular, to methods and systems for conducting highly enhanced three-dimensional reproduction and analysis of three-dimensional images and diagnosis and treatment related thereto.
The present application claims priority to two (2) provisional applications identified as follows: U.S. Application Ser. No. 60/196,208, filed Apr. 11, 2000; and U.S. Application Ser. No. 60/253,974, filed Nov. 29, 2000. Each of these earlier filed provisional applications are incorporated herein by reference.
It is known in the art of image diagnostic systems and methods to scan objects found in a host patient, retrieve signals resulting from such scanning, and process the signals to obtain information for possible diagnosis and treatment. To date, much of the effort has been directed to treatment of such data in two dimensions, and analysis has to a certain extent been constrained by two-dimensional limitations. Consequently, professionals relying on such information have, similarly, been somewhat constrained in their ability to create diagnostic and possible treatment models.
As a result of the inventors herein having the perspicacity to think and analyze based on three-dimensional vision, new methods and corresponding systems are now available for use in various applications. In one of the more basic applications, detection and analysis of objects in the body, especially growths, can be produced and analyzed to a degree of accuracy previously thought unknown. This is particularly useful especially in the area of life threatening pathologies, e.g., cancer. One particularly pernicious pathology is lung cancer. The ability to be able to detect early stage lung cancer has provided strong motivation to improve diagnostic capabilities.
For example, the small pulmonary nodule is a common radiographic finding that poses a diagnostic and clinical challenge in contemporary medicine. Approximately 170,000 new lung cancers are detected each year in the United States alone. Pulmonary nodules are the major indicator of these malignant lung cancers, but may also be signs of a variety of benign conditions. Of the all the new lung cancers currently detected each year in the United States, approximately 90% are fatal, responsible for nearly 160,000 deaths in 1999 alone [46]. In fact, lung cancer is the number one cause of cancer mortality, responsible for more deaths in men than prostate and colon cancer combined, and in women, for approximately as many as breast and colon cancer combined [46 ].
One contributing factor for this low survival rate is the historical lack of early detection, or the detection of the disease at a stage where it is most likely to be curable. Until now, the majority of lung cancers are either “symptom-detected,” or found on chest x-rays (CXR) of asymptomatic patients. In most cases, the tumor is detected at an advanced stage where opportunity for successful intervention is limited. It has now been shown, through the work of the Cornell Early Lung Cancer Action Project (ELCAP) that it may be possible to detect a significant number of malignancies at an early stage, where cure rates are much more promising, through the use of lung cancer screening [28]. The cornerstone of the ELCAP study is the use of computed tomography (CT) for the detection of small pulmonary nodules and the use of early-repeat CT (ERCT) for evaluation of these nodules, by high-resolution computed tomography (HRCT). It has been reported that low-dose CT has the potential to detect tumors 4-6 times as frequently as CXR [28]. It is theorized that this may engender a “stage-shift” in tumors detected by CT versus CXR. This additional time for potential intervention holds the promise of increasing survival from the current 12% to over 70% 5-year survival following resection of a stage I cancer [28, 76].
The tools developed in this work concern the application of computer vision and image analytic techniques to provide quantitative information describing the characteristics of pulmonary nodules seen in helical CT studies. They are an effort to capture in a reproducible, mathematical form the somewhat subjective characteristics traditionally used by radiologists and other physicians to describe these lesions in order to determine their likelihood of malignancy. Advances in helical CT technology are now allowing for the detection of a significant number of pulmonary nodules less than 1 cm in diameter. At this point, traditional methods for nodule characterization (many based on hand-measurement and subjective evaluation) become less effective, given the small size of these lesions. Through the use of computer-aided diagnostic techniques, we have a unique opportunity to assist the radiologist in the increasingly difficult task of diagnosing these small nodules. Furthermore, as the expected widespread acceptance of CT screening for early detection of lung cancer increases, computer-aided tools should play a prominent role in the early detection, characterization, and cure of lung cancer in many thousands of patients around the world.
Pulmonary nodules, while most important when they represent malignancy, are frequently the manifestations of a variety of benign conditions such as hamartoma, benign adenoma, or rarely, sarcoidosis, amyloidosis or infection. Fungal pulmonary infections (histoplasmosis, cryptococcosis, coccidiomycosis), mycobacterial (tuberculosis) and other bacterial infections may also, infrequently mimic neoplasms [69].
Establishing a specific diagnosis is important, since surgery is usually not needed in the treatment of benign small pulmonary nodules (SPNs). On the other hand, SPNs may represent malignancy (cancer) of several types (squamous-cell, adenocarcinoma, large-cell, small-cell, bronchioloalveolar carcinoma, etc.) where expeditious removal by thoracotomy is usually indicated. For example, approximately 5% of patients with “small-cell” lung cancer are diagnosed after a chest radiograph (x-ray) shows an SPN. Currently, the definitive diagnosis is made after the tumor has been removed surgically. If the diagnosis is made before the operation, clinical decision making is facilitated since multiorgan scanning and inediastinoscopy to detect spread or metastasis are performed prior to thoracotomy [85]. In addition, very small pulmonary nodules may not be detectable in the ordinary chest x-ray or conventional CT and are visible in helical (spiral) CT examinations. Therefore, improvement in the available techniques for the detection (e.g. differentiation from vessels) and differentiating malignant from benign SPNs are needed and will result in clinical and economic benefits.
Small pulmonary nodules are usually identified on routine chest radiographs (CXR) in asymptomatic patients or on chest radiographs obtained for other indications. Most nodules have a benign cause, but bronchogenic carcinoma, metastases, and other malignant processes are common causes and must be excluded in the differential diagnosis. As they are potentially malignant, SPNs require expeditious evaluation. Techniques including computerized tomography (CT) have been used for the differential diagnosis of pulmonary nodules. Radiologic evaluation includes chest x-ray (CXR) or computed tomography (CT) for the detection of nodules, high-resolution CT (HRCT) for further characterization, and CT and magnetic resonance imaging (MRI) to evaluate the mediastinum, and extrathoracic structures.
The radiologic characteristics that differentiate benign from malignant pulmonary lesions include size, the presence of calcification in a “benign pattern,” the shape and surface characteristics, rate of change in size, clinical characteristics of the patient such as age and smoking history, etc. Meticulous assessment of the margin of the nodule is necessary. Useful characteristics for the differentiation of benign from malignant include, sharpness of margin, the presence of spicules, length of spicules, spicules extending to the visceral pleura, pleural tail sign and circumscribed pleural thickening. Even with current criteria however, the best sensitivity and specificity that can be achieved are 85% and 78%, respectively. Further improvements in diagnostic accuracy are needed since at the present time many patients with ambiguous findings are referred for early surgery [72].
Mediastinal CT is the preferred modality for examining the mediastinum in these patients while magnetic resonance imaging is used selectively, e.g. in patients with superior sulcus tumors who are candidates for surgery. The differential diagnosis is an important consideration, particularly in patients with small pulmonary nodules, due to the increased sensitivity in lesion detection, increased specificity and lesion (tissue) characterization (usually done with MILI) [29]. Metabolic imaging by positron-emission tomography (PET) provides many advantages over conventional anatomically-based cross-sectional imaging, although its use is generally resolution-limited to nodules larger than 1 cm in diameter. For routine use in oncology, a detailed assessment of specific efficiency of PET is indicated [67]. Mediastinoscopy is done using CT for guidance [61]. Fine-needle aspiration biopsy is a useful diagnostic tool [95]. Thoracic biopsy is a safe and accurate minimally invasive surgical approach to resectable peripheral SPN. Many patients require surgical biopsy especially those with adequate physiologic reserve (that is those who would be able to tolerate surgery) [53]. Also, video-assisted thoracic surgery (VATS) often allows one to accomplish the same goal as the comparable open procedure but with less morbidity and a shorter hospital stay. With continued development of instrumentation, increasingly complex procedures continue to be accomplished with this technique [37].
Currently the diagnosis and management of SPNs is based on the basic principle that every nodule must be regarded as potentially malignant until proven otherwise. Malignant nodules are surgically removed unless the patient is in such bad shape from other diseases that he or she can not survive the surgery or there is proof that the cancer (malignant tumor) originated elsewhere in the body and that the SPN is a metastasis. On the other hand, resection of a benign nodule carries a small surgical cost. The major benefit derived from resection of the benign SPNs is that the diagnosis is made by pathological examination and malignancy is excluded. In addition, once a small pulmonary nodule has been detected, the diagnosis should be made quickly to avoid spreading of the malignancy.
When a malignant cause cannot be ruled out, the patient's age, smoking history, and nodule size must be considered. For moderate and high-risk patients, an immediate and more invasive work-up is indicated [17]. Observation by serial radiographs or CT studies may be the appropriate course for patients who are at low risk for malignancy. When the probability of malignancy is low, the patient may be advised to have repeated radiologic examinations in order to determine whether the SPN is changing over time before the decision about whether to operate is made. Therefore, making the diagnosis without surgery either at initial examination or after consideration of repeated examinations would be of benefit [47].
In addition to primary pulmonary pathology, pulmonary nodules may be due to metastatic disease from cancers of the breast, stomach, pancreas, kidney, bladder, and the genitourinary tract. Computed tomography, especially helical CT, is probably the most sensitive imaging technique in the identification of pulmonary metastases, because it detects a higher number of pulmonary nodules compared with other techniques. In a population of patients presenting with multiple, solitary pulmonary nodules, or the absence of nodules, helical CT shows 30 to 40% of supplementary nodules when compared to conventional scanning techniques. Helical, or spiral, scanning is a relatively new computed tomographic technique based on the continuous acquisition of volumetric CT data during continuous x-ray beam rotation and continuous patient translation through the scanner at constant velocity. It has many advantages over conventional CT including rapid image acquisition during a single-breath-hold scan of the lung, and the ability to obtain axial image reconstruct ions at arbitrary and overlapping intervals, thus allowing the detection of small lesions that otherwise would be inconspicuous because of respiratory motion or slice averaging. This leads to better identification of small pulmonary nodules and to high quality multiplanar reconstructions that can be useful in the study of mediastinal lymph nodes and the vascular and tracheobronchial spread of lung cancer [10].
In addition to its use in standard radiographic studies, helical CT also enables the optimal use of contrast products in which the iodine is poorly concentrated to study the pulmonary vessels. The continuity, both anatomically and for the lesions obtained by spiral CT is such that it is now possible to apply to thoracic pathology techniques of multiplanar and three-dimensional reconstruction. If all the information is contained in conventional transverse imaging is shown slice by slice, not everything is perceived by the observer because the information is inconveniently presented, by deconstructing anatomical and lesional picture. In helical CT, reconstructions of the volume may be inspected from a coronal, sagittal, oblique or three-dimensional perspective, furnishing supplementary information to axial images alone [66].
Traditionally, measurements of pulmonary nodules have been made by the radiologist using calipers or a ruler on a chest radiograph. The goal was to make an approximate measure of the nodule size by estimating the diameter seen in the projection captured on the chest film. The sizes could be recorded, along with subsequent measures to establish whether the nodule was growing and at what rate. This practice has carried over into the reading of many CT examinations. Although computer-based workstations are sometimes available for the review of CT studies, the sequential images are often output in “hard copy,” film format and viewed on a light box. Here too, caliper measurements are the norm, with size being estimated from the image with the greatest cross-sectional area.
As a step toward precision and reproducibility of measurement, many digital review workstations provide “digital calipers,” that allow the radiologist to make measurements using a mouse. While a general improvement of caliper-based measurements on film, it should be noted that the estimate of nodule size is still observer-dependent, for the following reasons: (a) selection of an appropriate cross-section (maximal area) for measurement is variable; (b) nodule cross-sections may not be circular, prompting multiple measurements and averaging schemes; (c) the margin of many nodules is somewhat indeterminate; and (d) window and level settings can markedly change the apparent size of a nodule.
These factors suggest that a more automated assessment of nodule size would be beneficial to the unbiased, reproducible estimation of size and growth. A potentially more important point is that the measurements made by the radiologist are nearly always two-dimensional in nature. With computer-aided methods, the entire nodule volume may be measured with great precision, eliminating the problems of slice selection, non-circularity, and window and level distortion.
Computer vision is the study of techniques for the extraction of information from images. The images may be two-, three-, or higher-dimensional in nature, and are acquired by varying types of image sensors, including conventional 2D cameras, video microscopy systems, x-ray film digitization, and computed tomography systems. The underlying physics of acquisition also varies, including the absorption of visible light, infrared, x-ray, and the sensing of magnetic resonance. Each of these may be accomplished in a variety of geometries and projections.
Computer vision is closely allied with its sister disciplines: image processing and computer graphics. Image processing is the application of techniques for the improvement or modification of an image, generally so that it may be better perceived by a human observer. Computer graphics is concerned with basically the inverse problem of computer vision: generation of images based on an informational description.
Most of the techniques now popular in computer vision have been developed for industrial and military applications. Examples include product quality control, and detection of tactical targets in radar, sonar, and conventional images. More recently, computer vision has been applied to problems in biology and medicine. Automated analysis of cell morphology from microscopic images, analysis of anatomical structures radiographic images, and study of function using advanced imaging modalities are fast becoming key applications for computer vision methods.
One notable difference between computer vision in medicine and biology and traditional computer vision is that in industrial applications, the structure of objects being studied is frequently known a priori, and also often has a carefully described geometry. Biological structures, however, while they normally have a particular structure, exhibit a wide range of variation, especially in pathological cases. Even normal anatomical structures defy simple geometric description, as they are three-dimensional, complex structures that exhibit variation between subjects and also often have a dynamic appearance in the same subject. Growth over time, as well as motion during image acquisition make it non-trivial to obtain easily-modeled data of a particular structure, even in a single subject.
These subtleties motivate techniques that can characterize the size, shape, and other qualities (e.g. density) of a biological structure in a quantitative, reproducible way. This allows for the comparison of data between subjects, as well as the study of a structure in a single subject over time. Such measures are precisely what is needed to effectively study the progression of a pulmonary nodule from first detection to eventual diagnosis, and in many cases, during follow-up as well.
The purpose of computed tomography (CT) is to display the anatomy of a slice of the body by measuring absorption of x-rays as they pass through the body in many trajectories. Imaging usually is done in slices perpendicular to the axial (head to toe) direction. The resulting measurements are manipulated mathematically to obtain two- or three-dimensional displays.
X-rays are electromagnetic radiation (10−10-10−8 nm) that are produced when an electron beam strikes a metal anode, typically made of tungsten. The anode is heated during the generation of the x-rays and dissipation of heat becomes a limitation in long examinations. Only x-rays in one general direction are allowed to exit the x-ray generator by the use of a beam collimator made of lead shielding material. As they travel through matter, the x-rays are attenuated, i.e., their intensity (related to the number of photons per unit cross sectional area) is decreased by absorption or scatter. Absorption occurs through the photoelectric effect, where the x-ray energy is given to an electron that is freed from the atom. Scatter occurs where x-rays of higher energy (generated by electrons accelerated by higher voltages, having higher frequency) transfer some of their energy to an electron (the energy necessary to free it from the atom), while the remaining energy becomes a secondary photon (x-ray) that is transmitted in a new direction. Thus, absorption is useful in diagnostic radiology since it occurs in a known direction while scatter creates noise due to secondary x-rays of unknown direction being detected by the x-ray sensor(s).
CT and all other x-ray diagnostic examinations are based on the fact that different structures of the body absorb or scatter x-rays to a different degree. The fundamental principle is expressed by Beer's Law
                                          ⅆ            I                                ⅆ            L                          =                              -            μ                    ⁢                                          ⁢          I                                    (        1.1        )            describing the attenuation of intensity of electromagnetic radiation as it passes through a homogeneous medium. In this equation, I is the intensity of the radiation (number of x-rays, photons, per unit surface area); L is the length of the pathway, and μ, is the linear attenuation coefficient. This equation merely describes that the x-ray attenuation as it goes through a unit length of the medium is proportional to the intensity of the incident radiation or that each x-ray (photon) has equal probability of being absorbed while traveling through a given thickness of a given material as any other x-ray (photon). This differential equation leads to the following exponential relation describing the intensity I of radiation as it passes through a medium:I=I0e−μL  (1.2)where I is the transmitted intensity through a thickness L. This is derived easily from the fundamental definition of e
                                          ⅆ                          ⅆ              x                                ⁢                      (                          ⅇ              x                        )                          =                  ⅇ          x                                    (        1.3        )            and the associated chain-rule for differentiable functions of x
                                          ⅆ                          ⅆ              x                                ⁢                      (                          ⅇ              u                        )                          =                              ⅇ            u                    ⁢                                    ⅆ              u                                      ⅆ              x                                                          (        1.4        )            Therefore,
                                          ⅆ            I                                ⅆ            L                          =                                            ⅆ                              (                                                      I                    0                                    ⁢                                      ⅇ                                                                  -                        μ                                            ⁢                                                                                          ⁢                      L                                                                      )                                                    ⅆ              L                                =                                    -              μ                        ⁢                                                  ⁢                                          I                0                            ⁡                              (                                  ⅇ                                                            -                      μ                                        ⁢                                                                                  ⁢                    L                                                  )                                                                        (        1.5        )            
FIG. 1 illustrates Equation 1.2, where the intensity of the radiation passing through the material, I, is a function of the intensity of the incident radiation, I0, the length of the material, L, and the linear attenuation coefficient of the material, μ. This coefficient is determined by the density of the material, the atomic number, and the wavelength (or energy spectrum for polychromatic x-rays) of the x-ray radiation under consideration. Thus, knowing the linear attenuation coefficient in each voxel of the body would provide information on the density and atomic number of the elements at that location.
The process of creating images from x-ray absorption data along a variety of angles and translations is called image reconstruction, involving the solution of a set of equations relating the geometry of absorption data seen in each projection. A given CT projection is the result of x-ray traversal through objects of varying attenuation coefficients at varying distances. This relationship can be generalized asIn=I0e−μ1L1·I0e−μ2L2 . . . I0e−μnLn  (1.6)where I(In) is the intensity of the radiation emerging from the body and detected by the detector, μ1, μ2, . . . , μ3, are attenuation coefficients of the individual structures traversed by the x-ray and L1, L2, . . . L3, are corresponding distances traveled by the x-rays (lengths). Assuming equal lengths from x-ray source to detectors in helical CT (L, chosen at the time of image reconstruction), Equation 1.6 can be rewritten asIn=I0e−(μ1+μ2+ . . . +μn)L  (1.7)FIG. 2 illustrates the physical system in which Equation 1.7 applies. In this case, the incident x-ray beam passes through several materials of varying length and attenuation characteristics. The computation of the individual attenuation coefficients (μ1, μ2, . . . , μn) requires the solution of an equal number of equations obtained from the different projections acquired in the CT procedure.
The usual method for solution of the set of equations and the generation of the CT image is called filtered back-projection, based on the Radon transform. In this method, a uniform value of attenuation is projected over the path of each ray such that the total attenuation is proportional to the measured attenuation [14, 15, 6]. These values are stored as elements of one vector of the reconstructed matrix. The procedure is repeated for each ray sum of the CT while corrections are made for voxels traversed by the x-ray beam in an oblique fashion. The assumption that x-ray attenuation is uniform throughout the path of each x-ray beam (obviously an oversimplification) results in blurring of the resulting image. In spite of this limitation, filtered back-propagation is popular because of its inherent parallelism. It allows processing of the data obtained by each ray while data acquisition continues in other projections, dramatically improving computational efficiency. The blurring is decreased by increasing the number of projections and by convolution with a filter function (convolution kernel). The filtering functions depend on x-ray tube geometry, detectors, intended effects (sharpening edges thus enhancing spatial resolution at the expense of decreased density resolution versus more refined density resolution while sacrificing spatial sharpness, etc.).
The relative pixel values obtained by this image reconstruction process are not the absolute values of the linear attenuation coefficients of individual voxels of the body (μ1+μ2+ . . . μn). The CT numbers, called Hounsfeld Units (HU) in honor of Godfrey Hounsfield (the inventor of CT scanning [30, 31]), are related to the linear attenuation coefficients as follows:
                              CT          ⁢                                          ⁢          number                =                              K            ⁡                          (                              μ                -                                  μ                  ω                                            )                                            μ            ω                                              (        1.8        )            where μω is the attenuation coefficient of water, μ the attenuation coefficient of the pixel in question, and K a constant. The value of K must be large enough (e.g. 200-2000) to accommodate the accuracy of the scanner. For example, consider a CT scanner with a density resolution of ±0.5%, or ±1 in 200. In this case, a value of K=200 would be sufficiently high to encode density values, as they are typically recorded with integer precision. A larger value of K would expand the scale beyond the accuracy of the scanner.
Helical CT is a newer method of data acquisition in computerized tomography [83, 40]. Data acquisition is continuous during rotation of the x-ray source and the detector (mounted in a toroidal gantry) around the patient who is simultaneously moved in the axial (head to toe) direction through the rotating gantry at precise speeds from 1 mm to more than 10 mm per second. The resulting helical projections are used to reconstruct an anisotropic Cartesian space representing the x-ray absorption at each voxel in the scanned patient volume.
In commonly used instruments, the x-ray beam is collimated to a fan that defines the image plane at any given time, and the array of detectors travels in a circular path around (typically 360°) the patient. This approach provides the opportunity to obtain many projections (and therefore images) in a short period of time (e.g. in one breath-hold), and the collection of data in a continuous volumetric manner, rather than in slices. This allows reconstruction in any position as well as improved two-dimensional and three-dimensional views. The major practical advantage of spiral CT is the ability to cover complete anatomic regions in a single exposure [38]. Isotropic imaging, where spatial resolution is the same in all three directions, will eventually become a reality [39]. Helical CT has been found useful in the study of pulmonary nodules where optimally centered studies have been shown to improve accuracy [40, 88].
The most important parameters of a CT protocol relate to the image geometry, reconstruction resolution, and radiation dose. Pitch is defined as the ratio of the table speed in mm/revolution (assuming a 360° reconstruction algorithm) to the collimation width of the x-ray beam. For a given beam collimation, the table may be advanced at a speed that would move the table through one collimation width per revolution, or 1:1 pitch, or at a higher pitch, reducing the effective radiation dose and scan time at the cost of image quality. For example, a protocol with 10 mm collimation and 20 mm/revolution table speed would acquire images with a 2:1 pitch.
The reconstruction resolution is described by several parameters. The in-plane resolution (the image plane is perpendicular to the scanner axis), which is equal in the x and y dimensions, gives the resolution of a pixel in a single 2D reconstructed image. The axial resolution, or resolution in the z dimension, is determined by the slice spacing, or the physical distance between reconstructed 2D images. This spacing is a function of the scan collimation, pitch, and reconstruction algorithm. Furthermore, images may be reconstructed at overlapping intervals (e.g. 10 mm overlapping slices at 5 mm intervals). The axial resolution of conventional CT protocols is often 5-20 times more coarse than the in-plane resolution. This results in 3D voxels that are anisotropic (not all dimensions exhibit equal spacing).
Radiation dose is a function of x-ray tube current, typically expressed in mA, tube voltage, expressed in kV (peak measurements are given units of kVp), and scan duration. The tube current and voltage directly affect the quality of reconstructed images, as the signal-to-noise ratio seen at the x-ray detectors is proportional to the dose used. Improvements in detector sensitivity and filtered image reconstruction algorithms are continually reducing the minimum dose required to achieve acceptable image quality at a given scan geometry.
There are three types of CT examinations commonly performed in the study of pulmonary nodules. The first is a low-dose, low (axial) resolution screening scan, aimed at the detection of possible pulmonary nodules in the lungs. The second, following detection of a pulmonary nodule, is a standard-dose scan of the entire lung volume. The third (and primary focus of this work) is a standard-dose, high-resolution scan of the region containing the detected pulmonary nodule, used for analysis and characterization of the nodule itself. A description of the parameters typically used in the different scan protocols is shown in FIG. 3.
Current CT scanners impose a tradeoff between image quality, slice thickness, and x-ray dose. Many scanners have a gantry rotation speed of approximately 1 revolution/second. This implies that a breath hold on the order of 30 seconds to 1 minute is required to obtain a whole lung scan. More recent scanners, using multislice technology, have reduced this time to just a few seconds, helping to eliminate respiratory and other motion artifacts. It may be anticipated that future developments in CT detector and processing technology may make possible much more rapid and higher resolution scans such that future screening scans may be taken at high (≈0.1 mm) axial resolution. However, current CAD systems are designed for the constraints of today's scanners which suggest a two-resolution approach: a low-dose initial whole-lung scan followed (in some protocols) by a high-resolution focused scan of detected nodules.
Much research has been done relating to the manual detection and characterization of pulmonary nodules by radiologists. In the past 10-15 years, work in computer-assisted and automated detection has appeared in the literature. Until quite recently, the dominant imaging modality for the study of cancer in the lung has been the chest radiograph, or chest x-ray (CXR). With the increasing use of CT for the detection of these lesions, however, we are now seeing an increase in the use of automated methods for the detection of pulmonary nodules in thoracic CT studies. Some work has also been reported dealing with characterization of pulmonary nodules, but it is only quite recently that advances have been made in this area. As characterization of pulmonary nodules from CT scans is the focus of this work, the following sections provide a view of pertinent related issues in the study of nodules using both CXR, and CT, in the screening setting, and for nodule characterization.
The goal of lung cancer screening is to detect pulmonary nodules, establish their probability of malignancy at an early stage, and thereby improve likelihood of cure. The traditional detection of pulmonary nodules occurs by their appearance on a routine chest radiograph, or one taken for unrelated reasons. In the past 10-15 years, research has been done in developing computer-aided methods for detection of these nodules.
In the 1970s, the National Cancer Institute (NCI) sponsored three large screening trials to evaluate the benefits of lung cancer screening using chest radiography and sputum cytology. These were held at the Mayo Clinic, Johns Hopkins, and Memorial Sloan-Kettering. [52, 20, 19]. Although these studies found that lung cancer screening allowed for earlier detection of lung cancer, better resectability of tumors, and better survivorship of the surgery, the overall mortality in the screened and control groups were similar. Thus, lung cancer screening was not recommended as national policy.
Detection of pulmonary nodules has traditionally been done through visual inspection of chest radiographs. Since the 1970's, however, much research has been devoted to the automation of this process. Computer-aided detection and analysis systems for pulmonary nodules fall into several categories. Image processing techniques have been used to improve contrast and otherwise enhance candidate regions for the visual detection of nodules [43, 44, 77]. Systems have been developed for the automated detection of candidate nodules [50, 96, 93, 8, 59]. Some work has also been done on automated analysis of nodules in CXR images for the prediction of malignancy [71, 58, 87].
The preprocessing stage of several CAD systems for the detection of nodules in CXR involves histogram manipulation. Techniques have been explored using histogram equalization and high-frequency enhancement [59]. Several studies have described the use of image differencing, which is the subtraction of a “nodule suppressed” image from one that is “nodule enhanced,” to help reduce the influence of other anatomical structures appearing in the radiograph [23, 96, 44, 93]. This enhancement of CXR images is achieved using one or more matched filters designed to identify the round regions characteristic of many nodules. For example, a disk-shaped filter may be convolved with the image to enhance nodule-like regions. A ring-shaped median filter may be used for nodule suppression. Related work in preliminary segmentation of lung images has also been done in the areas of lung boundary delineation [91, 3] and rib identification [70]. These steps help eliminate regions of the image from consideration as potential nodules.
A second stage of histogram manipulation may be performed following the initial image preprocessing. Multiple thresholding of the preliminary images may be performed to identify the incremental contribution of each intensity band to candidate regions. In this way, suspect regions are iteratively enlarged (by decreasing a threshold) and analyzed [50, 8]. Computed tomography (CT) is fast becoming the modality of choice for the detection and analysis of pulmonary nodules. Studies in lung cancer screening with CT have been performed in Japan [41, 78] and New York [28].
There have been several prototype systems for performing complete, automated lung CAD [94, 22, 84, 7] (i.e., taking as input whole-lung scans and identifying nodules). Note that these systems involve low-resolution screening scans and therefore, may be able to detect small nodules (<1 cm) but there is insufficient information to characterize them. These early attempts employ a multiphase approach involving the following three stages: (a) identification of the lung regions in the CT images; (b) separation of candidate nodules from vessels within the lung regions; and (c) classification of candidate nodules.
For diagnostic classification of small nodules, a focused HRCT scan of the region-of-interest is obtained. Prototype CAD systems for these scans have been developed in which the region-of-interest is specified by the radiologist and just the separation and classification stages are automatic [63, 42].
The lungs may be separated from other structures in the chest by first performing some noise reduction through spatial filtering and then applying a simple threshold with background removal to the whole-lung scan. As an example, a single slice from a whole-lung screening scan is shown in FIG. 4 and the extracted lung region is shown in FIG. 5. The entire lung volume can be measured or visualized as shown in FIG. 6. A similar method was used by Giger et al. [22], while Brown et al. [71] used an anatomical model with fuzzy set membership based on several soft constraints including threshold, continuity and relative position. Toshioka et al. [84] use geometric filtering to avoid irregularities in the detected boundary. They also use a second method, basing their lung boundary on predictions from rib locations in order to identify the presence of very large nodules.
Further analysis of other thoracic structures is also possible. For example, the lumen of the trachea and the main bronchi may be traced by a simple seeded region growing algorithm; typical results for screening data is shown in FIG. 7. This figure illustrates the limitations of the approach on screening scans due to their low axial resolution; the bronchial tree and the vascular structure can be extracted to a much finer degree with higher-resolution scans. Three-dimensional region growing combined with tree-traversal algorithms has also been used in removal of the lung vasculature as a preliminary step to nodule detection [90, 16, 79].
Until recently, nearly all of the literature on the characterization of pulmonary nodules from radiologic images was concerned with the radiographic appearance of the lesions to a trained radiologist. Most of the measures of nodule size were made manually using calipers or a ruler on chest radiographs or hardcopy CT images. Measurements of shape and density distribution were also described in somewhat qualitative terms.
Some notable work on the characterization and subsequent classification of pulmonary nodules may be found in [26, 47, 82, 73]. The chief characteristics reported were size, smoothness of edge, presence and (for larger nodules) pattern of classification, spiculation, lobulation, cavitation, and the presence of a pleural tail.
Perhaps more important than shape and density characterization, the estimation of nodule aggressiveness as a function of growth rate has been studied for some forty years. Nodule doubling time, as computed from manual measures of nodule diameter, has long been utilized effective predictor of nodule malignancy [12, 56].
A variety of techniques have been used to characterize potential candidate regions for nodule detection in screening data. Measurement of object circularity is often helpful, as this metric may help exclude rib crossings and other structures [50, 44, 92]. Methods involving gradient analysis have also been described [50, 49]. These are based on the notion that the edge gradients of a small nodule-like object should primarily point toward the center of that object, unlike the case with ribs and many vessels.
Morphological operations have been explored for the detection of nodules in CXR [96]. These nonlinear filters impose geometric constraints on the candidate regions, allowing, for example, round objects (e.g. nodules) to be distinguished from longer, flatter ones (e.g. ribs, heart, etc.). Correlation-based template matching has also been used to identify candidate nodules, an idea related to matched filter enhancement [8, 59].
The selection of candidate regions may be refined by one or more automated decision-making steps. Both neural networks [93, 59] and rule-based [8] systems have been used in this capacity. Inputs to these decision-refinement stages come from one or more shape, edge, or histogram features.
One of the most important issues regarding the use of CAD systems for nodule detection is the effective detection rate. Most computer-aided schemes for the analysis of chest radiographs unfortunately have relatively low sensitivity. In addition there is frequently a high false-positive rate (very low specificity) in these systems. A typical result is 2-7 false-positives per image, with sufficient sensitivity (over 70%) [93, 8]. Thus, the choice of operating point on the receiver operating characteristic (ROC) curve for a particular CXR analysis technique may be quite challenging.
In addition to the detection of pulmonary nodules, computer techniques have recently been studied for the classification of nodules found in chest radiographs. Computation of density gradients as a measure of shape and surface irregularity has been used to distinguish benign from malignant nodules [71]. Fractal texture analysis has also been applied to the nodule classification problem with encouraging results [58, 87].
When assessing nodule growth for the prediction of malignancy, two or more CT studies, separated by a suitable interscan interval, are needed. It is theorized that it may also be possible to estimate the probability of nodule malignancy based on size, shape, and density parameters determined from a single exam.
In a screening setting, 2D features are frequently measured to refine the set of nodule candidates for detection. In this capacity, Giger et al. [22] used measures including perimeter, area, compactness, circularity, elongation, and location within the lung. Similarly, Toshioka et al. [84] used area, thickness, circularity, density mean and variation, and location. For small nodule characterization, however, HRCT studies are required (see FIG. 3) and methods for high-resolution characterization are being explored [42, 63, 51].
McNitt-Gray et al. [51] have explored two-dimensional (2D) shape metrics including aspect ratios, circularity, and compactness, as well as a variety of texture measures (e.g. correlation texture and difference entropy). Their study also employed bending energy as a measure of surface characteristics. Three-dimensional measures of surface characteristics were studied by Kawata et al. (42). Using data isotropically resampled to 0.33 mm3 voxels, they measured Gaussian and mean curvature as metrics of irregularity in nodule margin.
Another challenging problem in the study of pulmonary nodules is the analysis of non-solid nodules, or ground-glass opacities (GGOs) [18, 11]. These lesions are difficult to characterize using traditional size and shape metrics.
Patient demographic information has been used as an adjunct to radiographic analysis in the prediction of small pulmonary nodule status. Several patient characteristics have been shown to be highly correlated with nodule malignancy. These include advanced age and smoking history (pack-years) [81, 82, 47].
To study the pulmonary nodule using computer-aided techniques, we must formulate one or more models of what a pulmonary nodule is, in particular, with respect to its radiographic appearance. Pulmonary nodules are lung neoplasms that fall into two general categories: benign and malignant. The separation between these categories is generally one based on rate of growth. Most benign lesions do not grow, or grow much more slowly than malignancies. There are, however, some notable exceptions. Infections in the lung may exhibit extremely fast growth. Also, it has been suggested that there may be some indolent, slow-growing malignant species. These malignancies are by far the exception, however, and doubling times for most malignant nodules range from 30-400 days [47].
Pulmonary nodules appear radiographically as round, opaque lesions with density slightly more dense than that of water (˜0-100 HU). Their position in the lung and immediate surroundings, from an image-analytic perspective, differentiates them into one of the following categories: (a) well-circumscribed—the nodule is located centrally in the lung, without significant connections to vasculature; (b) vascularized—the nodule is located centrally in the lung, but has significant vascularization (connections to neighboring vessels); (c) pleural tail—the nodule is near the pleural surface, connected by a thin structure (“pleural tail”); and (d) juxtapleural—a significant amount of the nodule periphery is connected to the pleural surface.
The above categories describe the types of solid pulmonary nodules seen in CT images. A second class, known currently as ground glass opacities (GGO) have a significantly less uniform radiographic appearance and will be considered separately later in this work.
One of the best predictors of nodule malignancy is growth rate. Similarly, nodule size, is also highly correlated with malignancy [82]. Significantly larger nodules (>3 cm in diameter) are more likely represent lung cancer [82]. Overall size has therefore been used in characterization and classification studies [26, 82].
In addition to these benefits, volumetric measurement allows for the identification of anisotropic growth. It is commonly assumed that pulmonary nodules grow uniformly in all dimensions.
In 1956, Collins et al. [12] succinctly described the relationship of growth and cancer, as well as the need for quantitative measurements of tumor growth:
“The definition of cancer, its diagnosis and its prognosis all depend upon description of growth. To the layman a synonym for cancer is a ‘growth.’ There are no quantitative terms for the description of growth or growth rate in clinical use.”
Their seminal paper continues, providing some of the most important early discussion of tumor “doubling time,” a measure of the amount of time (usually expressed in days) for a tumor to double in volume. Nodule growth, and the rate at which it takes place, is perhaps the feature most predictive of malignancy. Nodule doubling time and calcification in “benign” patterns have been the only universally accepted diagnostic tools of thoracic radiologists in the differentiation of benign from malignant lesions.
Traditional measures of the size and shape of pulmonary nodules were made manually on chest radiographs using calipers. Nodule size was characterized by a measure of diameter (based on the assumption that nodules appeared roughly circular), or as a combination of several “diameter” measurements.
Assessment of pulmonary nodule shape has been based on subjective shape characteristics including notions of roundness, spiculation, lobulation, and sharpness of edge. Nodule surface characteristics are frequently used to help classify benign from malignant nodules. In particular, the presence of spiculation on the surface, as well as lobulation of the overall nodule shape, have been reported as more common in malignant nodules [47, 82]. As with measurement of nodule size, there have been difficulties in making reproducible shape characterizations, due to the lack of a precise mathematical basis and measurement technique.
Several techniques for measuring nodule surface characteristics have been described in the literature. Huo et al. [32, 33] used analysis of radial edge gradients (a technique originally developed for mammography) in two-dimensional CT images. Kawata et al. [42] used a method of curvature estimation based on 3D image gradient analysis. Such techniques for curvature estimation, and the prerequisite 3D gradient analysis and edge detection have been described in detail [5, 54].
Detection of pulmonary nodules in screening CT scans is challenging due to the desire to limit radiation dose and scan time, both of which result in a reduction of information. The need to scan the full lung volume in a single breath hold motivates the use of large intervals between images (10 mm). An additional reduction in radiation dose achieved through low tube current further affects image quality. With these constraints, computer algorithms for the detection of pulmonary nodules have traditionally focused on two-dimensional (2D) algorithms or serial 2D methods that examine connectivity between nodule candidates in adjacent slices.
Automated CAD systems for the detection of pulmonary nodules have been based on a multistage approach, as illustrated in FIG. 8. The main stages are: (a) preprocessing, (b) identification, and (c) classification. Preprocessing involves noise filtering and isolation of the lung region from the image data. Identification is the use of image filters and knowledge based reasoning to develop a set of nodule candidates from the lung volume being searched. Classification is the analysis of nodule candidates, normally as part of a rule-based decision system, to reject other structures (e.g. vessels) from consideration and to assess the likelihood that each remaining candidate is a true nodule. The classification stage in detection systems is sometimes extended to include the separation of benign from malignant lesions, however, this may only achieve limited results with the low-dose, low (axial) resolution studies common to initial screening exams. Characterization and classification techniques that lead to prediction of malignancy are generally performed using high-resolution CT (HRCT).
A number of automated CAD systems for the detection of pulmonary nodules in low-dose screening studies have been described in the literature [94, 22, 84, 7, 4].
A typical lung cancer screening protocol consists of a low-dose (140 kVp, 40 mA) CT scan of the full lung volume using 10 mm collimation, reconstructed at 5 mm intervals, with an in-plane resolution of 0.5 mm or better. Following reconstruction, the resultant voxels are highly anisotropic (0.5×0.5×10 mm). Although the in-plane resolution may be sufficient for the detection (and preliminary characterization) of many lesions, small pulmonary nodules (less than 10 mm in diameter) may appear in only one to three contiguous images. This anisotropy presents special challenges in distinguishing between small pulmonary nodules and fine vasculature.
Blood vessels may have either a linear or elliptical cross-section in a single CT image, depending on orientation. The more perpendicular to the image plane a vessel lies, the more circular the resultant image. Therefore, it is a non-trivial problem to distinguish between small pulmonary nodules and small vascular cross-sections as the low axial resolution of screening studies limits the three-dimensional information needed to fully characterize these objects. The problem is further exacerbated by the fact that nodules are frequently attached to the pulmonary vasculature or to the pleural surface, increasing their difficulty of detection as they may be considered part of either confounding structure due to the similar density values found in many non-calcified nodules.
The preprocessing stage identifies the region of the CT images that correspond to the lung volume. Algorithms have been developed to automatically detect the thoracic cavity, lung volume, tracheobronchial tree, and pulmonary vasculature.
The first step in most pulmonary CAD systems is to identify the lung boundary, thereby eliminating other thoracic structures from the nodule search region. This is typically accomplished using serial 2D gray-level thresholding techniques on each CT image. Spatial filtering may be applied to reduce the effect of noise on the boundary segmentation. Once a preliminary boundary has been determined, geometric filtering may also be used to eliminate discontinuities in the boundaries determined in adjacent slices, as well as to reduce the contribution of image artifacts.
An example of this procedure is shown in FIG. 4 through FIG. 6. FIG. 4 shows a single 2D image from a low-dose screening scan. FIG. 5 shows the lung region in this cross-section produced using the techniques described above. A three-dimensional surface-shaded rendering can also be produced, as is shown in FIG. 6.
Giger et al. [22] used histogram analysis to determine appropriate thresholds for lung segmentation. Toshioka et al. [84] used spline representation and curvature analysis to refine lung boundaries based on standard thresholding as well as the detection of rib locations. This enables the detection of juxta-pleural nodules, which can be excluded from the lung volume using simpler threshold-based methods. Anatomical models were developed by Brown et al. [7] as the basis for a knowledge-based system that utilized fuzzy set membership based on density, spatial locality, and position. These models were then used in the determination of lung, tracheobronchial tree, and thoracic cavity regions. In our studies, we have found that simple thresholding combine with geometric filtering has produced good results in identifying these regions.
Another preprocessing step in pulmonary CAD systems is the identification of the tracheobronchial tree and pulmonary vasculature. Segmentation of the airways may be accomplished by seeded region-growing techniques. An example of this algorithm applied to screening data is shown in FIG. 12. It is evident that the low axial resolution of anisotropic screening data limits the extent to which the bronchi can be followed. Tree-based algorithms that explicitly track the bifurcations of the bronchi and pulmonary vasculature have been described by Wood et al. [90]. Related methods based on morphological filtering [97] and fuzzy logic [103] as well as the benefit of airway subtraction to nodule detection have also been described [16].
Once preprocessing has eliminated regions outside the lung from consideration, the next step in nodule detection is the identification of nodule candidates from the remaining lung volume. This is typically done through the use of one or more gray-level thresholding operations, as the nodules are of higher density than the surrounding lung parenchyma. Yamamoto et al. [94] combined single and double-thresholding with connected-component analysis to identify nodule candidates that are within a selected density range and size. In addition, morphological filters, including disk and ring-shaped kernels as well as 2D morphological opening (“rolling ball filter”), were used to select candidates. Multiple thresholding based on histogram analysis has been used to select nodule candidates and differentiate them from vessels [22]. In this system, while increasing. the threshold, vasculature that bifurcates along the scanner axis will produce one or more “daughter nodes” in adjacent slices, thereby reducing the likelihood that the candidate is a true nodule.
The final stage of a pulmonary nodule detection system, classification, is the refinement of a set of nodule candidates based on size, shape, and other measures. A likelihood estimate that the remaining candidates are nodules is also often assigned. The techniques used in refining the set of nodule candidates are somewhat related to those used in the characterization and classification of nodules in high-resolution diagnostic studies. The difference, however, is that the aim of nodule candidate classification is to separate nodules from non-nodules, while characterization and classification in diagnostic HRCT is used to separate benign from malignant nodules. With this distinction in mind, many of the nodule feature metrics used in detection and diagnostic CAD systems describe similar size and shape characteristics. It is important to note, however, that the standard low-dose screening study is significantly anisotropic, and therefore largely limits the feasibility of performing measurements in three dimensions. For this reason, features used for nodule candidate classification have been primarily limited to 2D metrics or analysis of connectivity between adjacent 2D slices.
When a pulmonary nodule is detected at initial screening, high-resolution computed tomographic (HRCT) images may subsequently be acquired to further study the lesion. These studies typically involve scanning the region-of-interest (ROI) containing the nodule at a high in-plane resolution (0.3-0.5 mm), with small beam collimation (1.0 mm) and 1:1 pitch. At the time of HRCT, a diagnostic CT scan of the chest is also commonly done for use in patient management. FIG. 3 shows a comparison between low-dose screening, diagnostic, and HRCT studies.
An important consideration for HRCT studies is that the entire 3D volume of the lesion should be acquired without significant artifact (due to respiration or other patient motion). In traditional high-resolution studies, only the several images containing the largest 2D nodule cross-section were considered vital, as measurements would be made visually or with calipers. Thus, it is an object of the present invention to provide the capability for enhanced size, shape, and densitometric measurement of objects found in the body.